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Abstract 

' In evolutionary game theory, repeated two-player games are used to study strategy evolution 

Cs| I in a population under natural selection. As the evolution greatly depends on the interaction 

• structure, there has been growing interests in studying the games on graphs. In this setting, 

, players occupy the vertices of a graph and play the game only with their immediate neighbours. 

[JLh ' Various evolutionary dynamics have been studied in this setting for different games. Due to the 

QQ , complexity of the analysis, however, most of the work in this area is experimental. This paper 

aims to contribute to a more complete understanding, by providing rigorous analysis. We study 
the imitation dynamics on two classes of graph: cycles and complete graphs. We focus on three 
well known social dilemmas, namely the Prisoner's Dilemma, the Stag Hunt and the Snowdrift 
Game. We also consider, for completeness, the so-called Harmony Game. Our analysis shows 
that, on the cycle, all four games converge fast, either to total cooperation or total defection. 
^ I On the complete graph, all but the Snowdrift game converge fast, either to cooperation or 

defection. The Snowdrift game reaches a metastable state fast, where cooperators and defectors 
coexist. It will converge to cooperation or defection only after spending time in this state which 
is exponential in the size, n, of the graph. In exceptional cases, it will remain in this state 
indefinitely. Our theoretical results are supported by experimental investigations. 
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O ■ 1 Introduction 

Game theory is used as a mathematical tool to analyse strategic and economic decision-making 
by rational agents [11]. Evolutionary game theory, on the other hand, studies the evolution of 
strategies in these situations by natural selection. Thus it models the adaptive behaviour of 
I agents who meet repeatedly. Different strategic situations can be modelled by simple games, for 

a ■ example repeated 2x2 games with appropriate payoff matrices (see, for example, [4l[T8]). This 

is the setting we consider here. 

In this context, there are two players, each having two pure strategies (Cooperate and Defect). 
Here we study symmetric games. A game is said to be symmetric if it puts each player in the 
same strategic situation. Each player's payoff depends on the strategies chosen by the player and 
the opponent. The payoff obtained when both players cooperate is denoted by R (the reward 
for mutual cooperation). The payoff gained when both defect is denoted by P (the punishment 
for mutual defection). Finally, T (the temptation to defect) is earned by the defector and S (the 
sucker's payoff) is earned by the cooperator when one player defects and the other cooperates. 
Hence, the payoff matrix is: 



Cooperate 
Defect 



Cooperate Defect 

R S 
T P 
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Assuming strict ordinality of the payoffs (i.e. all four payoffs are distinct) gives rise to 12 
different strategic games that are symmetric [4l. In order to select the most relevant games, 
some more realistic assumptions are made about the payoffs. First, it is assumed that unilateral 
defection is preferred to unilateral cooperation (i.e. T > S), thus there is an incentive for non- 
cooperative behaviour |18| . Furthermore, for a dilemma to arise, the following conditions need 
to be satisfied [7]: (1) mutual cooperation is preferred over mutual defection (i.e. R > P), 
(2) mutual cooperation is preferred to unilateral cooperation (i.e. R > S), and (3) unilateral 
defection is preferred over mutual cooperation (i.e. T > R) ot mutual defection is preferred over 
unilateral cooperation (i.e. P > S). All these assumptions yield four strategic situations. They 
are : T > R > S > P which represents the Hawk-Dove or Snowdrift game (SG); R > T > 
P>SorR>P>T>S which represents the Assurance or Stag-Hunt game (SH) [7]; and 
T > R > P > S which represents the Prisoner's Dilemma game (PD). In addition to these 
well known social dilemmas, another game is widely studied which models the situation where 
the interests of the players match. This is represented by the regime defined by the rankings 
R> S > T > P ^ and R> T > S > P This game is called Harmony game (HG). In this 
game, cooperating strictly dominates defecting so that the only possible equilibrium of the game 
is for both players to cooperate. As this equilibrium is Pareto-optimal, there is no dilemma in 
this game. 

We use the normalisation of [15] to simplify the analysis. That is, we normalise the payoff 
for mutual cooperation (R) to 1 and the payoff for mutual defection (P) to 0. Then, as done in 
[15], if we restrict < T < 2 and — 1 < iS < 1, the behaviour of all four games can be captured, 



with each game corresponding to a quadrant in the ST plane as shown in Figure 1(a) Note 



that the quadrant for SH includes only the version defined hy R>T>P>S which is the 
standard version studied in the literature (see, for example, [T] [131 [16] ) . Thus the other version 
defined hyR>P>T>Sis omitted in this study. 




Figure 1: Game domains. 



Understanding the evolution of cooperation among selfish agents is clearly an important 
and challenging task. Much effort has been put into achieving this using the PD game as a 
model [10 . When the population is mixed, where each player is equally likely to meet any other, 
natural selection favours defection over cooperation [H] . Hence, Nowak and May [TH] studied the 
impact of arranging PD players in a two-dimensional array and concluded that cooperators and 
defectors can coexist indefinitely. Since then, considerable attention has been given to studying 
evolutionary game dynamics in spatial settings. In these settings, players are arranged as the 
vertices of a network and can play the game only with their immediate neighbours. The impact 
of network on the emergence of cooperation has also been emphasised in j5J [T^l [H] . The way in 
which cooperation evolves in spatial settings is called network reciprocity [^ , where cooperators 
survive by forming a cluster and helping each other within the cluster so that the defectors at 
the border cannot fare any better. In this paper we consider two extreme cases of the spatial 
setting. First, we consider the cycle graph in which the impact of the topology on the evolution 
is strongest [11 . Second, we consider the complete graph which models a mixed population. 
Several previous studies have focused on these types of graphs (see, for example, [Tl [51 [TTl [TS] ) . 
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In evolutionary game theory, the payoffs are regarded as the Darwinian fitness During 
the evolution, strategies earning higher payoffs become more common in the population. Inher- 
itance and imitation are two mechanisms by which successful strategies may spread. Between 
the two, imitation gives the more practical dynamics [S] p. 86]. Many versions of imitation 
have been studied in this context. Nowak and May |10j studied the imitation rule known as 
unconditional-imitation or imitate-the-best. Here, each player imitates the neighbour earning 
the highest payoff among the immediate neighbours and himself, in each round of the game. 
Furthermore, one of the three update rules studied in is the asynchronous version of the 
proportional-imitation [3, p. 87] rule. Under this rule, in each round of the game, a random 
individual is given a chance to update his strategy. The individual then chooses a neighbour 
uniformly at random and imitates the neighbour with some probability proportional to the 
payoff difference. (A simpler version of this update rule is called imitate the better [3, p. 87], 
in which an updating individual always imitates the randomly selected neighbour, but only if 
the neighbour's payoff is higher.) The synchronous version of this has been studied in [TH [15] . 
In this rule, each individual updates his strategy at the end of each round of the game in this 
fashion. ([M] refers to this as the replicator rule.) Finally, a stochastic combination of both 
versions of the proportional-imitation rule has been studied in |13) . Among the variations of im- 
itation update rule, proportional-imitation rules perform optimally, both from the individual's 
perspective and from the perspective of the population as a whole [IT]- In this paper, we will 
study the synchronous version of the proportional-imitation rule. In the rest of this paper, we 
frequently refer to this rule just as the imitation update rule. 

Most of the previous work on imitation rules has been empirical. For instance, the imitate- 
the-best rule on a two dimensional grid was studied in [TU] using simulations; the synchronous 
proportional-imitation rule on different types of graphs was explored in |14[|15| using simulations; 
and both of these imitation rules were investigated in [13] using simulations. The reason for 
the lack of rigorous analysis is that a vast number of possible patterns of strategies can be 
generated [9]. The empirical results give insights into the evolution, but some of the results 
cannot properly be understood without theoretical underpinning. On the theoretical side, the 
asynchronous version of the proportional-imitation-rule on the cycle was analysed in IllJ using 
fixation probabilities. The fixation probability is the probability that a population adopting the 
same strategy is overrun by a single individual adopting a mutant strategy. Although the results 
presented in [11] are interesting, the analysis based on fixation probabilities has two weaknesses. 
First, it does not show what happens when mutants invade in pairs, triples, etc. Second, it does 
not reveal any information about the rate of convergence to cooperation. 

Here we will study the synchronous proportional-imitation rule on cycles and complete graphs 
rigorously. Similar rigorous results were given for the Pavlov or Highest Cumulative Reward 
rule in the Iterated PD game (see [U [8] for details). Here, we make no assumptions about the 
initial configuration. We then calculate the time it takes for a steady state to be reached in 
these settings. By doing so, we provide rigorous support for the experimental results observed 
by [15] in complete graphs. In addition, we do a similar study for the cycle. Interestingly, this 
simple type of graph gives evidence that there are graphs on which cooperators and defectors 
cannot coexist for any of the four games, except for some specific payoffs values (i.e. T = S -\- 1 
and T ~ (S + l)/2). Furthermore we provide supporting simulation results for both types of 
graph. 

The outline of this paper is as follows: Some preliminaries are described in Section [2] Sec- 
tion |3] investigates the dynamics of the imitation on the cycle, while Section [4] investigates the 
same on the complete graph. Empirical results appear in Section [5j The impact of our results 
is discussed in Section [6l Finally, concluding remarks are presented in Section [71 

2 Preliminaries 

Let G — (y, E) be a connected graph with vertices V = {vi : i — 0,1, . . . ,n — 1}. Each player at 
the vertex i (0 < i < n—l) has a state Si G {0, 1}, where represents defection and 1 represents 
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cooperation. In addition, * is used as the don't care symbol for the states when the particular 
value of the state does not matter in the discussion (e.g. -k 1000). 

We will now define formally the proportional-imitation rule 3 with synchronous update. 
According to this rule, in each generation, each vertex vt {Q < i < n) plays the game with all 
its A/i neighbours, and stores the accrued payoff for that generation as Pi. At the end of the 
generation, all vertices simultaneously update their strategies as follows: each vertex Vi chooses 
one of its neighbours u.a.rQ (say Vj) and copies the strategy of Vj with probability 

p., = max < 0, — > , 

[_ ma J 

where a = max{r, 1} — minis', 0}, and m — max{A/i, Afj}. The denominator ma is a scaling 
factor which ensures that pi € [0, 1]. Here, pi is called the switching probability of Vi. Note that 
if Pi > Pj, Vi keeps the same strategy, i.e. the switching probability pi = 0. Clearly, the value 
of TO is 2 for cycles and n — 1 for complete graphs. 

We are interested to find the absorption time which is defined as the time required for the 
system to reach a steady state. The absorption time is determined in terms of the number of 
generations it takes for absorption as a function of the number of players, n. All-cooperate 
(where all players cooperate) and all-defect (where all players defect) are clearly two steady 
states under the imitation update rule. And there may exist other steady state configurations, 
where cooperators and defectors can coexist. Once the system reaches any of these states, it 
will remain there forever. 

3 Imitation on the cycle 

Suppose that a minimum of three agents occupy the vertices of a cycle graph G — (V, E), where 

y = {wj : i = 0, 1, . . . , n - 1} and E =^ {{v^,Vt+i} : i ^ 0,1, . . . ,n ~ 1} . 

Here and throughout this paper, addition and subtraction on the vertex subscripts is performed 
modulo n. We now introduce some terminology. Let S S {0, 1}^ be given. A cooperator-run 
(resp. defector-run) in S is an interval where < i, j < n, such that Sk = 1 (resp. 0) for 
i < k < j and Si^i = (resp. 1), Sj+i = (resp. 1). We use c-run to denote the cooperator-run 
and d-run to denote the defector-run. It is possible to have j < i, since we are working modulo 
n. Clearly all runs are disjoint. 

Suppose that TZ^ = is a d-run where the subscript d stands for defectors. The length 
of the run TZd, denoted by £{TZd), is the number of defectors in the run, i.e. £{Rd) = j — * + 1 
mod n. We will refer to a d-run of length £ as an £d-Tmi. A l^-run will also be called a singleton 
defector and a 2d-run will also be called a pair of defectors. There are two outer rim edges 
associated with TZd, namely {i — and + 1}. We use similar notations for c-runs, but 
with subscript c, which stands for cooperators. For example, £c-i'un represents a c-run of length 
£. A run, c-run or d-run, is said to grow if its length increases. A run whose length cannot 
be reduced to zero through any combination of updates is called a barrier. A run is said to be 
deleted if all its vertices change from defector to cooperator or vice versa. 

The main results for the game on cycle are listed below. The proofs are presented in the 
subsequent sections. 

Initial configuration: We assume that the initial configuration for the game is random: each 
vertex on the cycle is independently assigned as a cooperator with constant probability Pc and 
as a defector with probability 1 — Pc at the beginning of the game. Then, the following theorems 
give the absorption time for the n-vertex cycle. 

Tiieorem 1. // the payoffs are such that S + I < T , then the game converges to the all-defect 
state in time 0(n) with high probability. 

uniformly at random 
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Figure 2: Switching probabilities for w^, provided Wi+i has been chosen by Vi for imitation. 



Theorem 2. // i/ie payoffs satisfy 

Case I: ^ < T < + 1 and T > 25', or 

Case II: ^1 < T < 2S, 

then the game converges to the all-cooperate state in time 0{n) with high probability. 
Theorem 3. // the payoffs satisfy 

Case I: < T < and T > 2S , or S < T < 2S and T < or 
Case II: T <S, 

then the game converges to the all-cooperate state in time O(logn) with high probability. 

Remark 1. In this paper, an event Yn which depends on the size of the graph n is said to happen 
with high probability, or in short w.h.p., if Pr(y„) ^ 1 as n — >■ oo. 

3.1 Analysis 

Suppose, without loss of generality, that the random neighbour chosen by Vi for imitation is 
Vi-i-i. Then, the switching probabilities of Vi for different values of Si and Si+i are given in 
Figure [5] Clearly, we can ignore the two cases where the updating vertex and the randomly 
chosen neighbour already follow the same strategy, hence imitation has no effect. Ignoring these 
cases and expanding the formulae in Figure[2]with the possible values of Si-i and 5i+2, we obtain 
the results in Figure [3l This figure also shows the variable names we use to denote these different 
probabilities. Now, it is obvious that the switching probability of Vi, when it randomly chooses 
Vi-i-i for imitation, depends on the states iSi_i, 5^, iS^+i, and 5^+2- For notational convenience we 
write these states as Si-i[Si]Si+iSi+2 (e.g. 0[1]11), enclosing the state of Vi in square brackets. 

Note that the prerequisite for a player to switch strategy through imitation is to have at least 
one neighbour with a different strategy. Hence, the strategy changes on the cycle can happen 
only at the vertices of the outer rim edges of a c-run or a d-run, since all other vertices incident 
on a run have both their neighbours employing the same strategy as theirs. So, it is sufficient 
to focus our analysis on the borders of runs. 

For the analysis, we first need to know whether the values for the p.^'s are zero or not. As 



mentioned earlier, each game corresponds to a quadrant in the ST plane as shown in Figure 1(a) 
Yet, we will not use this game-based categorisation in our analysis. This is because, although 
these games have different (a, 5, T) values, they share the same behaviour in some cases, as a 
result of having the same sign for Pi — Pj : 

• r > ^ for PD and SG 
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Figure 3: Switching probabilities for Vi for different possible neighbour-states. 



• T>2S for PD and SH 

• T > S- for PD, SH and SG 

Instead, we categorise the game domain into six regions based on the dynamics for the 
purpose of analysis, as shown in Figure 1(b) with labels A, B, C, D, E and F. Each of these 
regions is characterised by its boundary conditions. They are: 



• Region A : T > S + 1 

• Region B : ^ < T < + 

• Region C : ^ < T < 25* 



1 and T > 25" 



Region D : 25* < T < 



s+i 



• Region E : < T < 25 and T < ^ 

• Region F : T < S 

Next we prove a lemma that will be used frequently in the proofs. 

Lemma 1. Let it be the length of a d-run at time t. If the length of the run reduces by at least 
p in expectation for some constant p > 0, the probability that the run is not deleted in 



p(l-e) 



generations is at most e ^(i e) fgr any e > 0. 

Proof. Let T be the time at which £t becomes for the first time. Let Xt be the decrement of 
the length of the run at time t. Thus 



iftt^ 0, 

1 otherwise. 



Thus, for t < T, the decrements are independent random variables with expectation E[Xt] > 
p where Xt G {0,1}. Thus 

t 

(t^eo-Y.^^ ■ 

For any t <T, using the Chernoff bound, we get 



Pr^X»<pi(l-£) <exp(-i£V), 



(1) 



for any e > 0. Since £7- = 0, we have 



r 



> 



Hence, applying ([T]) for T steps, we get 



Pr r > 



p{l 



< cM-h^^pT) < cxp (-le^p-^ 



exp 



□ 
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Next we explore some properties of the initial random configuration. 



3.1.1 Properties of the initial configuration 

In this study, the initial configuration is assumed to be generated randomly as follows: each 
vertex on the cycle is independently assigned as a cooperator with constant probability Pc and 
as a defector with probability pd = 1 — Pc at the beginning of the game. Hence, at the beginning 
of the game, the expected number of cooperators present on the cycle is npc- So, it is reasonable 
to assume — Q.{\/n), so that there will be some cooperators at the start. 

Lemma 2. The probability that there is no k^-run, where k > \ is a constant, on the cycle at 
the beginning of the game is at most e~P' L"/*;J , 

Proof. Let Pk be the probability that /c-consecutive vertices are not all cooperators. Hence 

Pk^{l^pl) ■ 

There are [n/fcj disjoint /c-segments on the cycle. As each vertex is assigned the initial 
state independently, these segments are independent. Hence, the probability that none of those 
segments is all-cooperators is 

p\n/k\ ^ (1 _pfe)L"AJ < e-pJL»/feJ^ 

and the lemma is proved. □ 

Lemma 3. The following statement holds with high probability. The longest c-run generated in 
the initial configuration is of length Alogn for any A > such that Alog(l/pc) > 1- 

Proof. Let ri be the number of c-runs of length £. So, 

'E[ri] = npj . 



If £ = Alogn, then 



So, 



E[rf] = npc^ = np. 



Alogn _ ^l->'log(l/Pc) 



Pr(rf ^0) < n'^->^^°s{-L/P.) _ 

Since n^-^^°s('^/p^) — > as n — > cx) for Alog(l/pc) > 1, we do not expect to see any runs of 
length A log n or longer and the lemma is proved. □ 

Lemma 4. The following statement holds with high probability. The longest d-run generated in 
the initial configuration is of length Alogn for any A > such that Alog(l/pd) > 1- 

Proof. The proof is entirely similar to that of Lemma [31 so will be omitted. □ 

Lemma 5. Consider a segment on the cycle having d-runs of any length separated by Ic-runs. 
Such a segment must have a c-run of length greater than 1 at each end. Now, the following 
statement about this segment holds with high probability. The length of the longest such segment 
in the initial configuration is O(logn). 

Proof. The length of the segment can be calculated using a simple random walk. Suppose the 
vertices are assigned as cooperators (or ones) or defectors (or zeroes) from vq to Vn-i- At any 
time t, there can be four combinations of present and future assignments. They are: 00, 01, 10, 
and 11. Consider these as four states of a Markov chain M., and denote these states by sq, si, S2, 
and S3 respectively. The transition probabilities for these states are shown in Figure HI 
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1-Pc 



Figure 4: A random walk representing tire generation of initial configuration, modelled as a four- 
state Markov chain. The edges of the state diagram are labelled with transition probabilities. 



We are interested to find the total length of the d-runs connected by Ic-runs. Thus, we need 
to find the time to return to S3, or equivalently, the time to go from S2 to S3. This time is 
exactly the length of the d-runs separated by singleton cooperators. 

Clearly, from any other states of Ai , there is a probability at least of reaching S3 in two 
steps. Hence the probability that S3 will not be reached in 2t steps is at most p^*. Thus the 
probability that there exists any such run of length Alogn, for any A > 0, is at most 

„pAiog« < if A > 



n log(l/pc) 

Hence, if A > iog(^/p ) ; the probability of finding this special configuration of length Alogn 
tends to as n — > 00. Thus, with high probability, the maximum length of the segment is 
O(logn). □ 

Lemma 6. The following statement holds with high probability. The longest sequence of al- 
ternating I's and O's in the initial configuration is at most 2Alogn, for any A > such that 
A(log(l/p,) + log(l/pd)) > 1. 

Proof. Let ri be the number of occurrences of a chain of length £ having alternating I's and O's. 
Suppose i is even, without loss of generality. Then we have 

1 1 
E,[ri] = npc2pd2 . 

If £ = 2A log n, then 

E[r£] = npc^'°^>d^'°*^" = j^i-Aiog(i/p<,)-Aiog(i/pd) _ 
So, if A(log(l/pc) + log(l/pd)) > 1, we have 

Pr{re ^ 0) < „i-Ai°g(i/Pc)-Aiog(i/pd) ^ q as n 00. □ 

3.1.2 Emergence of defection 

In this section we prove that the defection emerges fast for the games in the region labelled A in 



Figure 1(b) This region, having S + 1 < T, covers the whole PD domain and half the domain of 
SH and SG. The table below shows which of the switching probabilities are zero for the vertex 

Vi. 
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Here, no defector will ever become a cooperator, but cooperators can become defectors. So, 
the game converges to the all-defect state fast. The following lemmas prove this. 

Lemma 7. Suppose TZc is the longest c-run in the cycle when the game is started and let 
i{TZc) — 7- Provided S + 1 < T, probability that all-defect state is not reached in J(Y^^ 

generations is at most e ^ Jqj- Qjiy £ > 0. 

Proof. In synchronous updating, each vertex updates its strategy at the end of every generation. 
When S + I < T, the only vertices that imitate their neighbours are the cooperators at either 
end of a c-run. (Note that a Ic-run has only one such cooperator.) A cooperator at this position 
chooses a defector for imitation with probability i . Let p be defined to be the minimum of the 
four possible switching probabilities (see Figure [Sj. It can be easily verified that we have 

p = min{a, 6, c, d} = 6 . 

Then, an £c-run {2 < £ < n) reduces in length at either end with probability at least ^p and 
a Ic-run is deleted with probability at least p, in every generation. Thus, the expected decrease 
in the length of any c-run is at least b. The time it takes for the longest c-run to be deleted 
completely is precisely the absorption time, since shorter runs are deleted faster. The result 
then follows from Lemma [1] □ 

In the worst case, the length of the longest c-run (7) can be n. This means all players 
cooperate at the beginning, which is an absorbing state. Hence, the all-defect state is never 
reached. But, if 7 = n — 1, in view of Lemma [7l it takes ^^^^-^ ~ f generations for the 
convergence to defection. However, it was shown in Lemma |4] that, when pc < 1, the length of 
the longest c-run is 0(log7i). Theorem [T] combines all these results. 



Proof of Theorem [1} By Lemma |31 the longest c-run present on the cycle at the beginning 
of the game is of length Alogn w.h.p for any A > such that Alog(l/pc) > 1- Then, by 
Lemma [71 the probability that the steady state has not been reached in 

Alogn 

- = O(logn) 



6(1 



generations is at most n ^(i-e) for any e > 0, where b — 2a ^ ^'^'-^ ^ ~ maxjT, 1} — miufS*, 0}. 
For a suitable value for A, this probability tends to as n — cx) which completes the proof. □ 



3.1.3 Emergence of cooperation 

In this section, we prove that cooperation emerges fast in regions B, C, D, E, and F. Before 
analysing these, let us first look at some common characteristics shown by the regions B, C, D 
and E. Figure [S] shows which of the switching probabilities are zero for each of these regions. 



Note, from Figure 1(b) that the regions B, C, D and E satisfy T < 5-1-1. Hence, Lemma [5] 



holds for these regions. 

Lemma 8. In B, C, D and E, if a c-run TZc of length at least 2 is adjacent to d-runs of length 
at least 2 at each end, the c-run grows in length or remains as it is. The expected growth of the 
run in one generation is b. 
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(b) Region C 


Si 


Si+1 


Si-lSi-^-2 


00 


01 


10 


11 





1 


+ 


+ 





+ 


1 








+ 









(d) Region E 



Figure 5: Switching probabilities for B, C, D and E. 



Proof. The proof is based on the observation that the switching probabihty is positive for 0[0]11 
and zero for 1[1]00 in the regions in question. 

Now, suppose TZc — such that i{TZc) > 2. Suppose TZc is bordered by at least two 
defectors on both sides, so we have ★OOllOO*. The vertices of the outer-rim edges of i?c, namely 
Vi-i,Vi,Vj and are in states 0,1,1 and respectively. Possible changes to these vertices 

after a generation are: 

• Vi-i, which is a defector itself, has another defector on its left and a cooperator on its 
right. Hence, switching can happen only if Vi-i tries to imitate from its right. As Fig- 
ure [3] indicates, this happens with probability (as the right neighbour is chosen with 
probability ^ and the actual switching happens independently with probability b which is 
non-zero according to Figure [5]). This makes TZc longer in length by 1. 

• Vi, which is a cooperator itself, has another cooperator on its right and a defector on its 
left. Hence an effective imitation can happen only when it copies from its left neighbour. 
But, as shown in Figure [Sj the probability of switching to defection in this scenario (i.e. 
1[1]00) is zero. Hence, this vertex will not change its strategy. 

• By symmetry, Vj does the same as Vi. 

• By symmetry, Wj+i does the same as Vi-i. 

Hence, TZc grows in length with some non-zero probability. The expected growth of TZc at 
either end is ^b in one generation. Hence the total expected growth of the run equals 6. □ 

Remark 2. Lemma [8] implies that an £c-run {£ > 2) bordered by at least 2 defectors is hard to 
eliminate in B, C, D and E. We say "hard" because, as we will see later, one such configuration 
can be eliminated by another in some regions. By combining this observation with the dynamics 
of each region separately, we later establish the conditions determining a barrier for each region. 

Now we investigate the dynamics of a singleton cooperator having at least two defectors at 
its either end. 

Lemma 9. A singleton cooperator (Ic-run) bordered on both sides by at least two defectors (i.e. 
★00100*j can grow up to length 3 in C and E, whereas it is deleted in B and D. 

Proof. Let Rc = be a Ic-run and let Si-2 — Si-i = Si+i = 5^+2 = 0. Here, Vi might 
imitate both its neighbours and has 0[1]00 in both cases. As the comparison tables in Figure [6] 
show, the switching probability for 0[1]00 is positive in B and D, and zero in E and C. Hence, 
Vi might become a defector in B and D, but will remain as a cooperator in E and C. 

Next, let us see what happens to the neighbours of i?c, namely u^-i and Wi+i. It can be 
easily verified that both Vi-i and w^+i have a defector at one side and the cooperator Vi at the 
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Figure 6: The switching probabihties related to singleton cooperators 



other side. So, the switching can happen only when these vertices copy from Vi. In that case, 
the defectors appear as 0[0]10 which has a zero switching probability in B and D and a non-zero 
switching probability in C and E. 

To sum up, in B and D, while Vi switches to defection, its defector neighbours remain 
unchanged, hence Rc is deleted. But, in C and E, while Vi remains as a cooperator, its neighbours 
can become cooperators too, potentially increasing the length of the Ic-run up to 3. □ 

A singleton defector having longer c-run neighbours has the potential to grow in regions B 
and C, as the following lemma shows. 

Lemma 10. A singleton defector (Id-run) bordered on both sides by at least two cooperators 
(i.e. ★11011*J can grow up to length 3 in B and C, whereas it is deleted in D and E. 

Proof. In this case, the defector in the middle could imitate from both its neighbours and has 
the neighbourhood of 1[0]11 on either side, and the adjacent cooperators can copy only from the 
middle defector and have 1[1]01. The dynamics for these two cases are compared in Figure[7l 
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Figure 7: The switching probabilities related to singleton defectors 

As Figure [7] shows, in E and D, the switching probability is positive for 1[0]11 and zero for 
1[1]01. Consequently, in ★llOll*, the defector might become a cooperator while its neigbouring 
cooperators remain unchanged. Hence, the l^-run is deleted in E and D. But, in the case of 
B and C, the opposite is true: the middle defector will remain as it is while its cooperator 
neighbours switch to defection. Thus, the singleton defector might become a 2d-run or a 3d-run 
as claimed. □ 



Let us now analyse each region separately. 
Region B <T <S+l and T > 2S) 



Figure 1(b) shows this region with label B. Figure 5(a) shows which cases have zero and non- 
zero switching probabilities. In this section we prove that cooperation evolves in linear time in 
this region. Analysing the actual imitation process to prove this is complicated. Fortunately, 
we can use a simplified model for this purpose. The following lemma forms the basis for the 
simplification of the process. 

Lemma 11. An ic-run (£ > A) is a barrier in region B, whereas a c-run shorter than 4 might 
be deleted through a sequence of updates. 



Proof. In region B, 
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a Ic-run is always deleted: Lemma [3] shows that a Ic-rmi is deleted if it has two defectors 
on both sides. Now, when a Ic-run is adjacent to a Id-run on either side, as in ★10[1]01*, 
it is readily verified that the Ic-run will be turned into a defector whereas the adjacent 
defectors remain unchanged. Thus, a Ic-run cannot grow if it is adjacent to singleton 
defectors at both ends. Obviously this would be the case when a Ic-run has a singleton 
defector on one side and at least two defectors on the other side. 

It is noteworthy that if all the cooperators on the cycle exist as Ic-runs, the game converges 
to all-defect, since singleton cooperators can never survive. 

a 2c-run can grow or be deleted: A 2c-run grows if it is bordered by at least two defectors 
as shown in Lemma [H Now, when a 2c-run is adjacent to Id-runs on both sides, either of 
the Id-runs can grow, deleting a cooperator in the 2c-run, as illustrated in Lemma [TUl and 
removing the 2c-run completely. Thus, when a 2c-run is adjacent to a Id-run on one side 
and at least two defectors on the other, it has the possibility of growing or reducing to a 
Ic-run which is subsequently deleted. 

An interesting case shows that even a 2c run bordered by at least two defectors can be 
removed if it is adjacent to another 2c-run. Consider the configuration 0011001100. This 
might first become 0011101100, then become 001100010, and finally become 001100000, 
deleting one of the 2c-runs completely. 

a 3c-run can grow or be deleted: Like the pair of cooperators, although a 3c-run can grow 
when it is bordered by two defectors at either end, there is a possibility of it being deleted 
if it has singleton defectors at both ends. 

an £c-run {i > 4) can never be deleted: Even if there are singleton defectors at either end 
of a 4c-run , the run's length will be reduced to 2 in the worst case. The resulting 2c- 
run will then be bordered by two defectors, hence will start growing again, as shown in 
Lemma ISl Obviously, longer runs are more stable and cannot be deleted. 

In short, the key observations are: Ic-runs are always deleted; 2c-runs and 3c-runs might 
grow or be deleted; and, runs of length 4 or more can never be deleted. Hence a run of length 
4 or more is a barrier. □ 

Clearly, the worst case for the evolution of cooperation is when there is only one minimal 
barrier (a 4c-run) at the start of the game. We use this worst case scenario to determine 
the absorption time. But, we need to address an issue before doing that. Although the time 
calculated in this way gives the worst case for a c-run to grow until the all-cooperate state is 
reached, it might not include the overhead time required if there are many runs. There are two 
types of such overheads: the time spent on handling short runs that can become a barrier or 
be deleted, and the time spent on merging two c-runs. We first calculate the worst case time 
for these overheads in Lemma [T^] and [13] respectively. 

Lemma 12. Let (pa denote the expected time it takes for a Ic, 2c, Sc-ran to be deleted or become 
a barrier (a run of length 4-)- Then (j)s = 0{1). 

Proof. Recall that a singleton cooperator can never grow. According to Figure [Sj it is deleted 
with probability 

• a if it is in the form 00[1]00, 

• c if it is in the form 10[1]01, and 

• i(a -I- c) if it is in the form 10[1]00. 

We also have a <c. Hence, the worst expected time it takes for a Ic to be removed is 1/a, 
since it has the geometric distribution with probability of success of a. Now it is enough to 
determine the time it takes for a 2c-run or a 3c-run to become a Ic-run or a 4c-run, as we know 
that Ic's are removed in constant time. For calculating this, consider a random walk on the 
number of cooperators, i.e. {1,2,3,4}. Then we need to show that the walk reaches 1 or 4 in 
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constant time irrespective of where it starts. Next we determine the corresponding probabihties 
of this process. 

Suppose the current position of the random walk is 2 or 3, i.e. there is a 2c-run or a 3c-run 
respectively. A 2c-run or a 3c-run can exist in three forms: 

• 001100 or 0011100. In this case, the probability of going to the right is 6 — ^b^ and the 
probability of going to the left is 0. 

• 001101 or 0011101. In this case, the probability of going to the right is ^b(l — ^J) and 
the probability of going to the left is ^^(l — 56). 

• 101101 or 1011101. In this case, the probability of going to the right is and the probability 
of going to the left is d— jcP. 

We want an upper bound on the time it takes to reach state 1 or 4. It takes longest to reach 
4 if the probability of moving to the right is the minimum non-null probability, r say. If the 
probability of moving to right is zero, the walk never reaches 4 and is absorbed at 0. We have 

r = min{6-i62 i6(l-i(i)} . 

Similarly, it takes longest to reach 1 when the probability q of moving left from 2 or 3 is 

q = min{d- ^cP , ^d{l - 56)} . 

So we have a Gambler's Ruin problem with absorbing barriers at 1 and 4. Using the standard 
results when r + q = 1 (see, for example, Feller[3J p. 345]), if the game is started from state 
£ S {2,3}, the expected duration of the process is 

r e 4 1 - {q/rY 

I — — — j- 11 r ^ q, 

De ^ < q - r q - r I - (q/rp 

[ii4:-e) ifr^q. 

Let -Dmax = max{£'2, D3}. Clearly Dmax is constant as it only involves the constants q and 
r which, in turn, can be expressed in terms of the constants S and T. There is a self-loop at 2 
and 3 with constant probability, at most 1 — {r + q), but this slows down the random walk only 
by a constant factor. Thus it takes only constant time to reach 1 or 4. We showed earlier that 
it takes constant time to go from 1 to 0. Hence the total expected time is constant. □ 

The other overhead of having more than one run is the time required to merge them. Two 
barriers merge together when they are separated by two defectors and both defectors switch 
to cooperation simultaneously. As the switching of both defectors happens independently with 
some probability, there is a possibility that only one of the two defectors switches to cooperation 
whereby a singleton defector is created between the two c-runs. Then, as shown in Lemma [TUl 
the singleton defector can grow up to length 3. And then the barriers start growing again. This 
is repeated until the runs are merged. The following lemma proves that the worst case merging 
time for two c-runs of length at least 4 is a constant. 

Lemma 13. Let <j)m denote the expected time it takes for two barriers (i.e. two c-runs of length 
at least 4-) separated by 1 or 2 defectors to merge together. Then <f>rn = 0(1). 

Proof. The merging process can be modelled as a simple absorbing Markov chain with states 
Mi (0 < i < 3) where i denote the number of defectors in between the barriers. Mq is the ab- 
sorbing state and all other states are transient. Using Figure [31 we can determine the transition 
probabilities for this Markov chain. The corresponding transition matrix in canonical form is 





Ah 




Ma 


M3 


Mo 


Ml 


- {l-d/2f 


d{l 


-6/2)2 


(?/4 





M2 


b{l - 6/2) 
62/4 


(1- 





62/4 


M3 


6(1 


-6/2) 


(1-6/2)2 





Mo 













1 



13 



Note that all transition probabilities are constants. Hence, using standard methods, absorp- 
tion time can be calculated. Let be the absorption time when the chain starts at Mi. Then 
we get 

165 - + 63 + led - 46J - hd? 



db^{4-b-d) 
2(86 - + b^ + 8d- 2bd - bd^) 
J62(4_5_d)_ ~ 
166 - 1262 _^ _^ _ 35^2 

db^{4-b-d) 



The worst case merging time 0m = max{Ti, T2, t^}. Since the time calculation involves only 
constants, is constant, and the lemma is proved. □ 

Now we will assume that when the game is started there is a 4c-run and an (n — 4)d-run on 
the cycle. The following lemma proves that it takes linear time for the 4c-run to grow up to 
length n — 2. 

Lemma 14. If the game is started with a Ac-run and an {n~A)d-run on the cycle, the expected 
time it takes for cooperation to spread to n — 2 positions, denoted by Tg, is 0{n) w.h.p. 

Proof. As Lemma [5] shows, an ic-inn {£ > 2) continues to grow as long as it is bordered by at 
least two defectors. As we assume that the game is started with only a 4c-run on the cycle, this 
run can grow up to length n — 2. 

Now, let Ti be an upper bound on the number of steps taken to go from i cooperators to 
(i + 1) cooperators on the cycle. Then, from Figure |3l the adjacent defectors at both ends of 
the c-run switch to cooperation with probability ^6. Hence, the probability of increasing the 

length by at least lisl— (l — ^6)^. Let us denote this probability by pi. Thus we have 

Pr(r, =t) = (l-pi)*-Vi • 
Ti has a geometric distribution with parameter pi. Hence, the total expected time Tg is 

(n — 6) 



E[T„ 



71 — 3 

E 

i=4 



Pi 



0{n) 



Let us now bound the probability of getting large deviations from the mean E[Tg]. From 
the definition of Ti, we obtain 



Pr(r, >t) < (1-pi)* <e 



-Pit 



Thus, for t 



13 log n 
pi ' 



we get 



Pr {T > 



P log n ^ 



< 



1 



Thus, deviations of size " are unlikely. In other words, Ti lies within the range 

/3 log n 



0, 

' Pi 

and let Y = J^^^i Then, Yi £ [0, 1] with high probability, and we have 



with high probability. Now, define a set of random variables Yi such that Yi — ^^j^^^'^ , 

probability, and we have 
PiE[Tg] _ n~6 



E[y] = 



/? log n (3 log n 



Since 1^4, Ys, . . . , Yn-2 are independent random variables taking values in [0,1], we may apply 
Chernoff-Hoeffding to get 

Pr(y ^ (l±e)E[y]) < 2e~ 
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If £ = , the following holds for sufficiently large n. 

Pr (r ^ (1 ± £)Em) < 2e-3/3i°g" = 4^ . 

It follows immediately that Tg lies within the range (1 ±£)E[Tg] with high probability. Thus 
we can conclude that Tg e -^^^-^ ± 0(V^log n), so Tg = Q(n) with high probability. □ 

Proof of Theorem [2] (Case - I ): In the imitation process, runs just grow or decrease 
in length and no runs are ever created. Decreasing in length might mean the removal of runs. 
When a d-run is removed, two c-runs are merged and vice versa. Thus, the worst case absorption 
time includes the following: 

1. Tg - the worst case time required for c-runs to grow as much as possible, i.e. the time taken 
for a single barrier to become an (n — 2)c-run which is 0{n) by Lemma [HI 

2. T„i - the worst case time required for merging c-runs. There can be at most 0{n) runs on 
the cycle. And the worst case time for merging two c-runs is 0(1) by Lemma Il3l Hence 
the total time spent on merging c-runs is 0{n). 

3. Ts - the worst case time required to handle the c-runs that are not barriers. This time is 
spent on growing shorter runs to become barriers or removing them. In Lemma fT2l it was 
shown that the time taken to handle one short run is 0(1). Thus, the time to handle all 
small runs is 0(n). 

Thus, the worst case absorption time T — Tg + Tm + Tg — 0{n), as claimed. Finally, it 
is worth emphasizing the fact that, in the actual process, these three different types of events 
happen simultaneously. But, we have added the times in order to get an upper bound. 

What remains is to show that there will be a barrier, a c-run of length greater than 3, at the 
beginning of the game. But it follows from Lemma [5] that not finding a barrier is exponentially 
unlikely, so the proof is complete. □ 

Region C (^ < T < 25') 

It is easily observed on Figure [S] that the only difference between the dynamics of region B 
and C is: singleton cooperators adjacent to at least two defectors can grow in C, but not in B, 
as proved in Lemma |9l Hence, the characteristics of C can be summarised as follows: 

• a Ic-run can grow or be deleted. 

• a 2c-run can grow or be deleted. 

• a 3c-run can grow or be deleted 

• an ^c-run {£ > 4) can never be deleted. 

Thus, ic-Tun {£ > 4) is a barrier in C too. Due to its similarity to B, the analysis of region 
B in Section [3.1.31 will be applicable for C, apart from the time required to deal with short runs 
which is determined in the lemma below. 

Lemma 15. Suppose (ps is the expected time it takes for a l^, 2^, ic-run to he deleted or become 
a barrier (a run of length 4)- Then, (j)g — 0(1). 

Proof. Technically speaking, Ic-runs, 2c-runs and 3c-runs perform a random walk before they 
become Oc-runs or 4c-runs. Hence, we consider a random walk on the number of cooperators, i.e. 
{0,1,2,3,4}. Now, it suffices to show that the walk reaches or 4 in constant time irrespective 
of where the walk starts. 

Suppose the current position of the walk is 1, i.e. there is a Ic-run. A Ic-run can exist in 
three forms giving rise to three different cases: 



15 



• 00[1]00. In this case, the probabihty of moving right is a — jo^ and the probabihty of 
moving left is 0; 

• 00[1]01. In this case, the probability of moving right is |a(l — ^c) and the probability of 
moving left is ^c(l — ^a); 

• 10[1]01. In this case, the probability of moving right is and the probability of moving 
left is c. 

Next, suppose the current position is 2 or 3, i.e. there is 2c-run or 3c-run respectively. The 
probabilities of movement from these states are the same as for region B. But we give them here 
for easy reference. A 2c-run or 3c-run can exist in three forms: 

• 001100 or 0011100. In this case, the probability of moving right is 6 — jb'^ and the proba- 
bility of moving left is 0. 

• 001101 or 0011101. In this case, the probability of moving right is ^b(l — iJ) and the 
probability of moving left is |rf(l — 

• 101101 or 1011101. In this case, the probability of moving right is and the probability 
of moving left is d— j(P. 

What we want is an upper bound on the time required to reach or 4. It takes longest 
to reach 4 if the probability of moving to the right takes the minimum non-null probability r. 
Hence we get 

r = min{a- ia^ {l - ^c) , b- {b^ , \b {\ ^ \d)] . 
Similarly, it takes longest to reach when the probability q of moving left is 

q = min |c, 5 c (l — la) , d — j(F , ^d (l — ^&)} • 

Now, this is simply a Gambler's Ruin problem with absorbing barriers at and 4. The worst 
case expected duration of the game can therefore be calculated as in Lemma [12] and shown to 
be 0(1). □ 

Proof of Theorem [2] (Case - II ): Recall that the absorption time 

T = Tg + Oin)q^m + Oin)q^„ 

where, Tg and are the same as for B, and 0^ is still 0(1) for C as proved in Lemma 1151 
Hence, T = 0(n) w.h.p. □ 

Regions D (25* < T < ^) and E (S* < T < 25 and T < ^) 

In regions D and E, a 2c-run is a barrier. The following lemma establishes this. 
Lemma 16. An ic-run (i >2) grows or remains unchanged, hence is a barrier in D and E. 

Proof. Lemma [5] proved that when a c-run of length at least two is adjacent to d-runs of length 
at least two, the c-run grows in length. What is remaining to be shown is, a c-run of length 
at least two is not deleted or reduced in length when it is adjacent to singleton defectors. This 
is a direct result of Lemma [10] which proves that when a l^-run is adjacent to at least two 
cooperators, the Id-run cannot grow in length in D and E. □ 

We will call the part of the cycle not containing any barriers the non-barrier. A non-barrier- 
segment is a set of vertices between two barriers. In region C, the non-barrier-segments have 
only defectors and singleton cooperators. The length of each non-barrier-segment decreases 
during the evolution since it has a barrier at either end which grows or remains unchanged. As 
the update is done synchronously, the length of every non-barrier-segment reduces in expecta- 
tion. The all-cooperate state is reached when the lengths of all non-barrier-segments reach 0. 
Obviously, the absorption time is dominated by the longest non-barrier-segment present at the 
beginning of the game. 
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Lemma 17. Suppose Red is the longest non-harrier- segment at the beginning of the game and 
let 7 be its initial length. In regions D and E, the probability that the all-cooperate state is not 

reached in ^^.^"^^^ generations is at most e ^(i-e) ^ Jor any £ > 0. 

Proof. Let x be the longest non-barrier segment, spanning from Vi to Vj. Let at be the length 
of X after t generations. Hence, the time t at which at becomes is the absorption time. 

Let us first determine the expected minimum negative growth of the x af its left end, i.e. 
along the edge {i — Lemma [TCI shows that ^c-run {£ > 2) never reduces in length. This 

means that Vi-i never switches to defection. So we determine the minimum probability that Vi 
becomes a cooperator. The switching of Vi depends on the status of Wi_2, fi-i, fi+i and 
Let 5* denote the status of the vertex Vi after t generations. Using Figure [51 the following table 
presents the possible values for the probability that Vi becomes a cooperator in D and E. 
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Hence the minimum probability that the left border Vi and, by symmetry, the right border 
Vj switch to cooperation is equal to minjifo, ^d} = ^d. Hence we have 

E[crt] < crt_i - d . 

This case is quite similar to Lemma [U and the result follows by a similar argument. □ 



Proof of Theorem [3| (Case - I): According to Lemma [Sj the longest chain of d-runs 
separated by singleton cooperators (non-barrier-segment) is 0(log7i) w.h.p. Substituting this 
value for 7 in Lemma [T7I shows that the probability that the all-cooperate is not reached in time 
^J'^i^^^ = 0{\og n) is at most 



6^0(logn) 

e" 2(i-£) 



6 ci log n 
2(l-£) 



'2(l-e) 



for any e > and a constant c\ > 0. So, for suitable values of ci and e, the above probability 
tends to zero as n — cx) . 

Hence, if there is at least one barrier (2c-run) at the beginning of the game, the game 
converges to cooperation fast. Furthermore, Lemma [2] shows that the initial configuration has 
a 2c-run except for exponentially small failure probability, completing the proof. □ 



Region F {T < S) 



This region has been labelled F in Figure 1(b) 
are given in the table below. 



The switching probabilities for this region 
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It is clear from the table above that the evolution happening in this region is the opposite 
to what happens in the region A (see Section 13.1.21 for details) . More precisely, defectors are 
imitated by cooperators in region A, while cooperators are imitated by defectors in region F. 
Hence, the cooperation evolves fast in this region and the following lemma holds. 
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Lemma 18. Suppose Rd is the longest d-run on the cycle when the game is started and let 
i{Rd) ~ 7- Provided T < S , probability that all-cooperate state is not reached in di'-e) 

generations is at most e 2(i-e) f^jiy s > 0. 

Proof. Proof of this lemma is similar to the proof of Lemma [T] Let p be defined as the minimmxi 
of the four possible switching probabilities. Then we get 

p = min{a, b,c,d} = c . 

In this case, a d-run of length ^ (1 < ^ < n) is reduced in length at both ends with probability 
at least ^p and a Ic-run is deleted with probability at least p. Also, the absorption time here is 
the time it takes for the longest d-run to be deleted. The rest of the proof is similar to that of 
Lemma [71 □ 



Proof of Theorem [3] (Case II - T < S") : The result directly follows from Lemma [T51 and 
Lemma HI □ 

A special case for B, C, D and E 

Consider the case where I's and O's appear in the cycle at alternating locations. This 
configuration will not be generated by the random initial configuration w.h.p. Because, as 
proved in Lemma [51 the longest chain of alternating I's and O's is O(logn) w.h.p. However, it 
is worth investigating this configuration as it yields some interesting outcome. 

Theorem 4. // the game ever reaches a state where every cooperator and every defector on the 
cycle exist as singletons, the following statements hold with high probability. 

1. the all-defect state is reached in time 0(1) with probability 1 in B and D. 

2. the all-cooperate state is reached, with probability strictly less than 1, in time O(logn) in 
C and in time 0{n) in E. 

Proof, n has to be even for this scenario to occur. So, let n — 2m. The configuration in question 
has alternating I's and O's throughout the cycle. In this case, all the O's appear as 1[0]10 having 
switching probability of while all the I's appear as 0[1]01 having switching probability of c > 
(see Figure [3]). If all the I's switch to in the same generation, then the all-defect state is 
reached in a single generation. However, the probability of that happening is (c)"*, which is 
small for large m since c < 1 . 

Now, consider the regions B and D. It is easily seen from their dynamics on Figure [5] that 
Ic-runs are always deleted. In a starting configuration of alternating I's and O's, there are only 
Ic-runs present. Although, they all appear in the form 10[1]01 in the initial configuration, the 
other forms 00[1]00 and 00[1]01 might be generated in the subsequent generations. Singleton 
cooperators are then deleted in the expected time of 1/a = 0{1) (see. Lemma [121 for the 
calculations involved). This proves the first part of the theorem. 

Now consider any 15 consecutive vertices in the initial configuration Wj-7 . . . Vj+7 such that 
Sj = 1. The probability that all cooperators in these vertices but Sj switch to defection together 
is (c)^(l — c). Expected number of such cases after the first generation is n(c)^(l — c). All these 
cases would have created a Ic-run that has at least 7 defectors on both sides. Recall that Ic-run 
can grow in C and E when they are bordered by at least two defectors (Lemma [9l). In the above 
case, once the Ic-runs grow to be a run of length 2 or 3, it still has at least 2 defectors at either 
side and can grow further (Lemma [8]). Thus, the required barrier for a guaranteed convergence 
to cooperation, i.e. a 4c-run for C and a 2c-run for E, will be created in time 0(1). Then the 
results follow from Theorem [21 and Theorem [31 It was remarked above that the game might 
converge to the all-defect state even in C and E with probability (c)™. So, the probability that 
all-cooperate is reached is less or equal to 1 — (c)™ < 1. □ 
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3.1.4 Borders 



We now analyse the behaviour of the games which he on borders between regions. On these 
borders, one might expect to see a mixed resuh of the two regions that the border separates. 
The resuhs below show this intuition is wrong. 

The Line T = S+1 

This is the border between regions A and B. Let Bab denote this border. The table below 
shows which of the switching probabilities are zero for Bab ■ 
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The following observations that can be verified using the table above will help our analysis. 

• A Ic-run is always deleted. 

• An i!c-run (i > 2) can never grow. 

• A Id-run can become a 2d or a 3d-run. 

• An id-Tun {i > 2) cannot grow if it is bordered by at least two cooperators. 

In essence, the only changes that happen in any generation are: a Ic-run is deleted, and a 
Id-run becomes a 2d-run or a 3d-run. This suggest that cooperators and defectors can coexist 
in a steady state as long as they are not singletons. 

Theorem 5. IfT~S + l, the game reaches steady state in time O(logn) with high probability. 
The steady state contains d-runs and c-runs of length at least 2, and the proportion of cooperators 
in the steady state is less or equal to the initial proportion. 

Proof. In the evolution phase, what happens is the elimination of singleton cooperators and 
extension of singleton defectors. We note the followings when we look at c-runs of different 
lengths: 

• An ^c-run {(■ > 2) having at least two defectors on both sides remain unchanged. 

• An ^c-run > 3) having at least two defectors on one side and a Id-run on the other side 
can be reduced in length by at most 1 and become stable as in the previous case. The 
probability that a cooperator is deleted in a generation is pi — ^d. 

• An ^c-run {£ > 4) having Id-runs at either sides can be reduced in length by at most 2 
and become stable as in the first case. The probability that a cooperator is deleted in a 
generation is at least p2 = ^d. 

• All other cases of c-runs are potentially deleted in the worst case. They are: 

— Ic-runs. In this case, the cooperator is deleted with probability at least = minja, c, 
i(a + c)} = a. 

— 2c-runs having at least one of its neighbour as Id-run. In this case, a cooperator is 
deleted with probability at least p^ = minjiJ, J— ^f?} = \d. 

— 3c-runs having both neighbours as Id-runs. In this case, a cooperator is deleted with 
probability at least p^ — d — j(P . 

Let pab nrin{pi,p2,P3,?'4,P5}- Hence, all cooperators are deleted with probability at least 
Pab hi any generation. Maximum number of cooperators that any c-run can loose is 3, thus the 
absorption time is the time it takes for a 3c-run to be deleted at the slowest rate. Then, by 
Lemma[Tl for any e > 0, the probability that a steady state is not reached in time ^ ^^(l-e) — ^i^) 
is a constant. Hence, after time O(logn), the probability that a steady state is not reached is 
at most n~^, for any k. □ 
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The Line T = ^ 

The Une T = separates the regions B U C and DUE. Let us call this Bbd- The 
corresponding switching probabilities are given in the table below. 
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It is readily verified from the above table that, on Bbd, the only way the defectors can 
spread is through deleting singleton cooperators. At the same time, no singleton cooperators 
are created during the evolution. Hence, deleting singleton cooperators is the only setback in 
the process of emergence of cooperation. In order to calculate the worst case absorption time, 
we divide the process into two phases. 

Phase I The singleton cooperators are allowed to disappear first, suppressing all other favourable 
developments. 

Phase II The rest of the evolution, assuming that there arc no singleton cooperators. In this 
phase, an i^-Yun {£ > 2) reduces in length until it is of length or 1. Singleton defectors 
cannot be deleted on Bbd- 

Note that, in the actual process though, both Phase I and II happen simultaneously. So the 
above gives an upper bound on the absorption time. First, we calculate the time required for 

Phase I. 

Lemma 19. All Ic-runs are removed in time 0(1). 

Proof. A Ic-run or singleton can exist in the cycle in three different surroundings, having a 
different probability of removal accordingly: 

• The Ic-run in 00[1]00 is deleted with probability a. 

• The Ic-run in 00[1]01 is deleted with probability ^(a + c). 

• The Ic-run in 10[1]01 is deleted with probability c. 

As the update rule is synchronous, the worst case time to remove all singletons is determined 
by the smallest probability of removal. That is, 

ipi = min{a, ^(a + c), c} = a, 

as we have d < c. Using the geometric distribution, the worst expected time required to remove 
all singletons is 

E[T,] = — = ^ = 0(1) . 

ipi a 

□ 

Next, we analyse Phase II. Here we analyse d-runs. As there are no Ic-runs present on 
the cycle in this phase, the length of a d-run will decrease until it is removed completely or it 
becomes a singleton. Since the updates are simultaneous, the time taken to reduce all d-runs is 
determined by the longest d-run present in the cycle. 

Lemma 20. Suppose Rd is the longest d-run on the cycle when Phase II is started and let 
£{Rc) =7- IfT = then, for any £ > 0, the probability that a steady state is not reached in 

generations is at most e ^(^~^) . In the steady state, cooperators and defectors coexist. 
The defector runs must be singletons and the cooperator runs cannot be singletons. 
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Proof. The key idea of this proof is, ah d-runs are bordered by a c-run of length at least 2 
in Phase II. Consequently, the vertices at either end of a d-run switch to cooperation with 
probability ■^b. Hence the expected decrease in length of the d-runs is equal to b. 

Also note that singleton defectors cannot be removed. Hence, all longer d-runs are either 
turned into a Ij-run or are removed. The longest d-run requires the longest time to delete 
or reduce to a singleton. When i?d is deleted, or made into a Id-run, the steady state will be 
reached. Let it be the length of i?d after t steps. Then we get 

E[£t] < £t-i~b = 7-6t . 

Now, the result directly follows from Lemma [T] □ 

Theorem 6. IfT^ steady state is reached in time 0(log7i) w.h.p. In the steady state, 

cooperators and defectors coexist. The defector runs must be singletons and the cooperators runs 
cannot be singletons. 

Proof. The worst case absorption time is obtained by summing the time required for Phase I 
and Phase II. Lemma fT9l shows the worst case time for Phase I is 0(1). Lemma [20l proves that, 
if T = ^Y^, the probability that a steady state is not reached in ^^^^^^ generations is at most 

e 2(i-e) fQj. £ > 0, where b = ^~^2a ^^'^ T ^^e size of the longest d-run at the beginning 
of Phase II. 

So we merely need to bound the length 7 of the longest d-run at the beginning of Phase II. We 
observe that the removal of singleton cooperators in Phase I will join d-runs together. However, 
we showed in Lemma [5] that the longest chain of defectors interleaved with singleton cooperators 
is still O(logn) w.h.p. Combining this with the estimate completes the proof. □ 



The Line T = S 

The line T ~ S on the ST-plane, denoted by Bfe, separates the regions F and E. The table 
below shows which of the switching probabilities are zero in this case. 
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Here, no cooperator ever becomes a defector. On Bfe, therefore, an Ic-iun {£ > 1) is a 
barrier. Yet, a Ic-run adjacent to a Id-run on either side cannot grow and help the convergence 
to cooperation. Therefore, a barrier for Bfe is an ic-Tun where £ > 2 or a Ic-run adjacent to two 
defectors. A non-barrier-segment is bounded by two such barriers. The non-barrier segments 
are eliminated by successively deleting the end vertices. 

There are clearly two types of non-barrier-segment for Bfe' a d-run, and a run of alternating 
I's and O's. In both cases, the all-cooperate state is attained by the expansion of the barriers. 

Lemma 21. Suppose Red is the longest non-barrier- segment and let 7 be its initial length. 
Then, if T — S , the probability that the all-cooperate state is not reached in ^^^"^^^ generations 

is at most e 2(i-e) ^ j'gj- Qj^y e > 0. 

Proof. The proof is very similar to that of Lemma 1171 so is omitted. □ 

What remains is to determine the length of the longest non-barrier-segment. Obviously, in 
the worst case, at the beginning of the game there will be only one barrier and the non-barrier- 
segment will be of length n — 1 . Then, by Lemma |21[ the worst-case absorption time is 0(n) 
with high probability. But, when the game is started with a random configuration, the longest 
non-barrier-segment is only O(logn). 
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Theorem 7. IJT = S, the all-cooperate state is reached in time 0{\ogn), with high probability. 

Proof. The longest d-run present at the beginning of the game is O(logn) w.h.p by Lemma |31 
and the longest chain of alternating defectors and cooperators is O(logn) w.h.p by Lemma [51 
Hence, the longest non-barrier-segment is 0(log n) w.h.p. Substituting 7 in Lemma [211 completes 
the proof. □ 

Remark 3. It is readily verified that alternating I's and O's throughout the cycle is a steady 
state for Bfe- But, this state can never be reached unless it is the initial configuration. 

The Line T = 25 

Bed is the line between the regions E and D. The switching probability table given below 
shows that the dynamics on this line are close to the dynamics of E. In fact, the only difference 
is that singleton cooperators, having at least two defectors adjacent, can grow in E, but not 
on Bed- Even so, we analysed the evolution of cooperation for E without considering that 
singletons can contribute to the evolution of cooperation. Only the growth of barriers was 
considered. Hence, the proof of Theorem [3] also applies here. 
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Theorem 8. On the line T — 2S, the all-cooperate state will be reached in time O(logn), with 
high probability. □ 

3.2 Summary 

We have proved rigorously that the games converge fast for all —1 < 5 < 1 and < T < 2. 
We have done this by grouping the games based on various relations between the payoffs. The 
results are summarised below. 

1. < T < ^ and T ^ 2S: This encompasses regions D, E, F and the border between E and 

F. In these regions, the all-cooperate state is reached in time O(logn). 

2. < T < S + I: This encompasses regions B and C. In these regions, the all-cooperate 
state is reached in time 0{n). 

3. 5* -I- 1 < r < 2: This contains region A. In this region, the all-defect state is reached in time 

O(logn). 

4. r = 2S: This is the border between BUD and CUE. On this line, the all-cooperate state is 

reached in time 0{n). 

5. T = ^Y^'- This is the border between B U C and DUE. On this line, a steady state is reached 

in time O(logn). Here cooperators and defectors can coexist indefinitely. 

6. T ^ S + 1: This is the border between A and B. On this line, a steady state is reached in 

time O(logn). Here cooperators and defectors can coexist indefinitely. 

The coexistence of cooperators and defectors when T = 5 + 1 can be explained as follows. 
When a defector earning T is adjacent to a cooperator earning 5 4-1, the switching probability for 
both players is zero (e.g. 11110000, where this applies to the middle cooperator and defector). 
There is a player following the same strategy on one side and a player following a different 
strategy, but with the same payoff, on the other side. Hence, there is no incentive for imitation. 
The coexistence on 2T = 5 + 1 can be explained in the same way. This happens when a singleton 
defector is between two runs of cooperators (e.g. 1110111). 
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Remark 4. As mentioned earlier, another version of the imitation strategy on the cycle was 
studied analytically in [TT , by calculating fixation probabilities. The fixation probabilities can 
be calculated easily using our arguments. Let Pc be the fixation probability for the cooperator, 
i.e. the probability that the game converges to cooperation when a single cooperator is added 
to n — 1 defectors on the cycle. Then we have Pc = for regions A, B and D; and Pc = 1 for 
regions C, E and F. However, the games converge to cooperation in all but region A when a 
maximum of four cooperators are introduced. This illustrates the drawbacks of analysis based 
on fixation probabilities. 



4 Imitation on the complete graph 

In this section, we analyse the imitation update rule on the complete graph if„. Each vertex 
of the graph plays the game with all other vertices. Hence, all cooperators receive the same 
accrued payoff in a given generation, and have the same switching probability. The same applies 
to the defectors. 

Now, let St denote the number of cooperators in K„ at time t. Let 7^^ and denote the 
total payoff obtained by a defector and a cooperator respectively, at time t. As before, using 
the normalisation P = 1 and P = 0, we get 

jyt 1 ^tT if X — d, 

\ {5t-l) + {n-St)S ifx = c. 

Let Ayx ='Py—Vl. where x,y € {c, d}. Figure [5] gives the values of l^yx for different values 
of X and y. Then the switching probability for a vertex adopting the strategy x at time which 
has chosen a vertex adopting y for imitation, is 

Px = max<^ 0, ) = max<^ 0, \, 

(n— l)a J { (n— l)a J 

where a = maxjP, 1} — minis', 0}. Here the denominator ensures that pi e [0, 1]. Note that 
only or can be positive at any t. 
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Figure 8: Payoff difference between two players adopting x and y at time t on a complete graph. 
Here, d stands for defection and c stands for a cooperation. 



Theorem 9. Let 9t denote the fraction of cooperators on the complete graph K„ at time t. 
Suppose, for large n, that 9q, S and T satisfy 

1 < mm<l, IH S 



and that there is at least one cooperator at the beginning of the game (i.e. 9o > Then, for 
any e > 0, the probability that the all-cooperate state is not reached in time 

1 ,j^g^n(l-.o) 



is at most e, where ip^ satisfies 

. (0o(l-S-T) + S I-T; , , . rr, r., 

ipd < mm < , > , with a = 1 — mmjS, 0} 

a a 
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Proof. Let St — nOt. A defector switches to cooperation in any generation with some positive 
probability only a cooperator with higher payoff is chosen for imitation. That is, a defector 
switches to cooperation at time t only if Acd > 0. Hence, from Figure [3 the required condition 
is 

Acd = 5t(l - 5" - T) + ?i5 - 1 > . 
Substituting nOt for 5t and rearranging, we get 

so, for large enough n, this condition is 

T < 1 + 11-^5 . (2) 

As defectors become cooperators, 6t increases. If S" < 0, inequality ([2]) holds as Ot increases up 
to 1. But, if iS" > 0, the inequality holds only until Ot = 1 provided n is large enough and 

T <1 . (3) 

Thus, if S and T satisfy inequalities ([2]) and ([3]) at f = 0, then 9t increases from 6*0 to 1. Let 
(^d > be a lower bound on the switching probability at any step during this process. That is, 

. , , 0t(l-S~T) + S . (9o(l-S-T) + S (1-S-T) + S 

(fid < mt = = mi" ' 

eo<et<i a 

where a = max{T, 1} — minj^, 0} = 1 — minj^*, 0}. Let 7^ denote the number of defectors at 
time t. Then, we can determine the expected value of 74 in terms of "ft-i- 

E[7t I 7t_i] < 7t_i - 7t-i(-^j-^Y^)</'d < 7t-i - 7t-i(- — ^^^Vd = 7t-i(l - Vd^t-ij ■ 

We know that, under conditions ([2]) and 0t > do for any t > 1. Substituting this in the 
above inequality, we get 

E[7t I 7t-i] < 7t-i(l - VdOo) 
Thus, by the rule of total expectation, we have 

E[7t] < (1 - ^ddoMjt^i] . 

Applying this iteratively for 1 to t — 1, we obtain 

E[7*] < (1 - ^ddoYMlo] - (1 - ^ddoYlo ■ 
So, for any e > 0, when 

we have E^ft] < £■ We know that any nonzero value of jt is at least 1. Using Markov's inequality, 
we obtain 

Pr[7t ^ 0] = Pr[7i > 1] < £[7*] < e . 
Finally, substituting 70 = n(l — 6*0) into (|4]) completes the proof. □ 

Theorem 10. Let 9t denote the fraction of cooperators on the complete graph Kn at time t. 
Suppose, for large n, that Oq, S and T satisfy 

T > 1 + ^5, and 5* < 0, 
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and that there is at least one defector at the beginning of the game (i.e. (1 — Oq) > Then, 
for any e, the probability that the all-defect state is not reached in time 

1 , fnOo 



is at most e, where ipc satisfies 

. f T-in-l)S eo{T + S-l)-S \ 

ipc < nim \ — 7 ^ , r , with a ~ maxj J , 1 [ — d . 

1^ [n— l)a a J 

Proof. This can be proved in the same way as Theorem [HI Hence, only the main differences are 
highhghted here. 

Let 5t = nOt as earher. For a cooperator to switch to defection at time t, the switching 
probability p* has to be positive. This is possible only if Adc is positive. Hence, from Figure |51 
we have 

Adc = -(5t(l - S -T) -nS + l> Q . 
Substituting 5t — nOt into the above inequality and solving for T yields 

For large enough n, this is 

T > 1 + ^^- (5) 

As cooperators become defectors, 9t decreases. Inequality ^ continues to hold during this 
process if 

5* < . (6) 

So, if inequalities ^ and ^ hold initially, the game converges to defection. The proof can be 
completed by estimating Ei[dt] in terms of Sq, as in TheoremlHl □ 

Corollary 1. Let T,S, and 6q he such that the cooperators and defectors receive equal payoffs 
at the start of the game. That is, 

T = 1 + — ( S ) — S*, or equivalently 5o 



T + S-l 



Then all players maintain their initial strategy indefinitely. 

This is the line in the ST plane which divides the regions where the all-cooperate state is 
reached fast (see Theorem\§j) and the all-defect state is reached fast (see Theoreml 1 0\) . 



Next we deal with the remaining region. This corresponds roughly to the Snowdrift game. 

Theorem 11. Let St denote the number of cooperators at time t. Then, the following statements 
fail with probability exponentially small in n. If 6q is outside the range 6* ± for any 

constant e > 0, 5 > and T > 1, then St reaches this range in time O(logn) where 

r* nS - I 

S* = 



(S + T-l) 



Thereafter, if 5* is an integer, St becomes S* in time O(lognloglogn) and remains with St = S* 
forever. If S* is not an integer, then either the all-cooperate or the all-defect state will eventually 
he reached, after n*^'"^ time. 
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Proof, li St{l — S—T)+nS—l > 0, defectors in it'n become cooperators with non-null probability 
while the cooperators remain as cooperators. Note that we have S + T > 1. Hence the above 
inequality can be rewritten as 

nS — 1 

Similarly, if the following inequality is satisfied, the cooperators become defectors with non- 
null probability while the defectors remain as defectors. 

J. nS — 1 
dt > 



S + T-1 

Finally, it can easily be verified that if 6t — S* , the switching probabilities of both cooperators 
and defectors are zero. In short, 5t oscillates around S* for a given S and T as follows: 

• if St < S* , then 6t+i > St 

• if (5f > (5*, then dt+i < St 

• if (5t = (5*, then, St+i — S* . Note that this can only happen if S* is an integer. 
First, suppose St > S* . Then, we have that St — St+\ is binomial Bin((5t,pf), where 

n^StSt{,S^T -\)-nS+\ 

Vt = 7fi 7T • (7) 

n — 1 J (n — 1) 

So we have 

, . . n-St (5t(5' + T-l)-n5+l 
E(5t+i = St- St . (8) 

n — 1 i (n — 1) 

Similarly, for the case where St < (5*, we have (5t+i — St is Bin(n — St, pt), where 

St -{St{S + T-l)-nS+l) 

Pt = 7 7T , (9) 

n — 1 1 [n ~ 1) 

and 

Trrx 1 s: / J ^ St{S + T - 1) - nS + 1 

E[St+i] = St-{n-St) — . 10 

n — 1 I [n~ 1) 



It is clear from (jH]) and (|T0|) that the drift towards S* is symmetrical. The rest of the proof, 
therefore, considers only the case where St > S* . Now, let At be the number of cooperators with 
binomial distribution Bin(At,7t), where 

At{S + T-l)-nS+l , , 

^* ^ T{^) ■ ^''^ 

Let / denote the interval S* ± ri^/^+^ for any constant e. Suppose A,- ^ S^ > S* + v}!"^^^ . Let 
T = min{< > t | (5r G /}. Then, the process S-r . . .St is stochastically dominated by A,- . . . At 
since we have 74 > pt. We now prove that the probability that St becomes less than S* is 
exponentially small when t <T. Let /3t — At / {n — 1), and 



so (fTT|) becomes 



where 



(n-l)(5 + r-l) S + T-1' 



It = (A-r)^^^l^ - iPt-ns, (12) 



{S + T-1) 



T 

so s is bounded away from and 1. Also we have 



E[/3t+i] ^ Pt - /3t{/3t - I3*)s . (13) 
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Let A* = {n~ l)l3*. While At > A*, we have E[At+i] > A* since 

E[At+i] - At-At(A-/?*)s> A* 

holds whenever /3ts < 1 which is always true. Let (1 — a;)E[A(_|_i] = A*. Then, from and 
using /3* < ft < 1, we obtain 

A* 

a; = 1 - 

= 1 - 



At - At{(3t - I3*)s 

ft-ft(ft-/3*)s 
^ ft - /3* 1 - Pts 
ft 1-fts + fts 

>(l-s)(ft-ft) = ^i^(ft-ft), (14) 



and 



At- 


-At(ft- 


-ft)s 


(n- 


l)(ft- 


ft2s + /3,fts) 


(n- 


l)(ft- 


fts + ftfts) 




l)ft(l - 


-s + fts) 




l)ft(l 


- s + S/T) . 



(15) 

Now, using the ChernofF bound, we have 

Pr(At+i < A*) = Pr(At+i < (1 - x)E[At+i]) < exp ( - ix2E[At+i]), 

so when ft > /3* + using and (ITSI) we get 

Pr(At+i < A*) < exp ( - an^') , (16) 

which is exponentially small for any constant e > 0, where a — /3*(1 — S')^(l — s + S/T)/{3T'^). 
Stochastic domination then implies that, when St > 5* + n^^'^'^'^ , 5t decreases steadily, except 
for some exponentially small probability. By symmetry, we have that if (5t < i5* — n^^'^'^^ ., St 
increases steadily, except for exponentially small probability. 
Let at — 5t/{n — 1), and 

{nS - 1) S 



{n^l){S + T -I) S + T-V 

so ([7]) becomes 

pt = (l-at + l/(n-l))(at-a*) ^^+^~^^ = ft(at-a*), (17) 

where 

(l/(n - 1))^ <qt<{l-<7* + l/(n - 1))^ 

so qt is bounded away from and 1. (Similarly, we have pt = zt{cf* — at), where < zt < 
a*{S + T -1)/T <1.) Also dH) becomes 

E[fTt+i] =at- at{l -at + l/(n - l)){at - a*){S + T- l)/T . (18) 

We suppose ao = 1 (i.e. n — 1 defectors at i = 0). Next we determine the time it takes for at 
to drop into the range a* ± n"^/^"*""^. 
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First, consider at > ^(l + cr*). Then ([TH]) imphes 

E[(Tt+i] < at- X{l-<Jt + l/{n-l)) < at~X{l~at), (19) 

where A = cr*^(5+r-l)/2T. We have immediately E[(Ti] < l — X/{n—l). Assume by induction 
that E[crt] < 1 - A(l + A)*-7(n - 1). Then, from (HH), 

n — 1 n— i n— 1 

continuing the induction. Thus, at time to — O(logn), we will have E[crtJ < i(l + ct*). Then 
implies 

E[at+i-a*] < at-a*-Ai(CTt-(T*) = (l - n)iat - a*), (20) 

where /x = (1 - (t*)(t* (5* + T - l)/2r. We have E[crt„ - cr*] < i(l - cr*), and to = O(logn), so, 
at time ti ~ O(logn), we will have E[(TtJ < cr* + for any constant e > 0. We will also 

have at J > cr*, except for exponentially small probability, from (1161) . 

When t > ti, we require more careful analysis. As crt+i can be greater or less than a*, we 
will consider the quantity (crj+i — ct*)^. That is, 

E[(at+i -a*)2] = E[(at+i-E[at+i])2] + (E[(7t+i]-a*)'. 

Here, E[(cr(+i — E[crt_|_i])^] can be determined from the variance of Bm{St,pt) if (Jt > a* , and 
from the variance of Bin(n — 5t,pt) if Ut < a*, by scaling. Below we use the fact that the 
variance of Bin(TO,p) is mp(l — p) < mp. Now, using ([T7]), (PO)) and that the drift around a* is 
symmetrical, we get 

E[{at+i-a*f] = E[{at+i-n<^t+i]f] + {n'yt+i]-<J*y 

^ iT? ^ + (1 ^ /")^(o"t - cr*)^, where = (5(9* or (n - St)zt 

(n — 1)'^ 

< ^-^^ + ii-,nat-ar 

[n-l) 



V n - 1 cr, - cr* / 



~in-l)\at 
< {I - fi){at - a*)\ 

for some constant cq, provided 

Wt-cr*\ > - ci/n, 

[n - l)/i(l - /i) 

for some constant ci. Observe that this applies both when at > a* and at < a* . 

Thus, at time ^2 = O (log n), we wih have either jcri^ -cr* | < ci/n or E [(erf ^ -cr*)^] < c^/n''+^, 
for any constant k. In the latter case, using Markov's inequality, we have 

PT{\at,-a*\>ci/n) = Pr {{at, - a*f > cl/n^) < 1/nK 

So, in either case, we will have jcrt^ — cr*| < ci/n with probability at least 1 — for any k. 

Thus, at t = t2, we have |crt — cr*| = cil{n — 1), for some constant C2 > 1. Then we have 
\bt-{n- \)a* \^{n-\)\at-a*\^C2. Also 

C2(l-<T* + l/(n-l))(5 + T-l) C3 

Vt - 



(n-l)T 71-1 

for some constant C3. So 8t — (5t+i is approximately binomial Bin((n — l)cr*, 03/(71. — 1)), which 
is approximately Poiss(c3cr*) . 
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Suppose {n — l)a* is an integer £. Then Pi{St+i — £) is a constant. If this occurs, we 
have pt — 0, so the process wiU remain at St = £ forever. If it does not occur, we will have 
\St+i — £\ — O(logn) with probability at least 1 — n~'', for any k. Thus, after a further time 
O(loglogn), we will again have that Pi{5t+i = ^) is a constant. After O(logn) repetitions of 
this, we will have Pr{6t — £) = I — , for any k. The total time for this to occur will be 
0(logn log log n). 

If (n-l)(T* is not an integer, then |crt-cr*| = f}(l/n) for alH. So Pr (Jt+i e {0,n}) = n^c-^"), 
for all t. Thus, after time n'-"^"'\ we will have 5t+i e {0, n\ with high probability. □ 

Remark 5. li 5* < n/2, the game eventually converges to defection with higher probability, 
whereas, if 5* < n/2, the game eventually converges to cooperation with higher probability. To 
see this, suppose S* < n/2. Then, due to the symmetry about 6* , we have 

Pt[St = 0] = Pt[St = 2(5*], 

at time T. Moreover, even after reaching 26*, it still takes exponential time for 5t to reach n, 
the all-cooperate state. Hence, the game will reach the all-defect state with higher probability. 
Similarly, the game will reach cooperation with higher probability when S* > n/2. 

5 Simulations 

Since our results are largely asymptotic, we have also simulated the imitation update rule on 
the cycle and the complete graph. The results are presented in this section. 

5.1 Cycle graph 

Figures [9] and [10] show the results obtained for the cycle. Each data point in these figures 
represents an average of 100 repetitions. The simulations were run for 100 x 100 different values 
of S and T, with an initial configuration having cooperators uniformly distributed on the cycle. 

Figures [9] shows the fraction of cooperators present on a cycle of length 100 after 10000 
generations. When the game is started with 10% cooperators, the game converges to the all- 
defect state if T > 5-1- 1 (region A); whereas, to the all-cooperate state if T > (S*-!- 1)/2 (regions 
C, E, and F). These results agree with our analytical results in Section [3.1.21 and Section fB.l.SI 
Now, in the region where 2S < T < S + 1 (regions B and D), the average cooperators after 
10000 generations is around 60%. Closer examination of the results shows that this is because 
the game reaches the all-defect state fast around 40% of the time and the all-cooperate state fast 
around 60% of the time. This is not surprising, since when the cooperators proportion is as low 
as 10%, the required barriers might not be present at the beginning of the game. More precisely, 
recall that region B needs 4 consecutive (4c-run) cooperators for a guaranteed convergence to 
cooperation. When n — 100 and Pc — 0.10, the expected number of such barriers present at the 
beginning of the game is very low (0.01). In region D, the smallest barrier is two consecutive 
cooperators (2c-run). Here again, the expected number of barriers is low (1.00). 

This raises an interesting question: how can the game converge to cooperation all the time 
in C and E, which also need a barrier of 4 and 2 consecutive cooperators, respectively? The 
answer is simple. In C and E, a singleton cooperator (Ic-run) can spread if it has two defectors 
adjacent to it. However, we did not consider a singleton cooperator as a barrier in our analysis 
in Section TS.l.SI Because, even in these regions, when a singleton cooperator (Ic-run) is adjacent 
to a singleton defector (l(j-run), it might become a defector. 

With this explanation, it is not surprising that, when the game is started with 25% cooper- 
ators, the convergence towards the all-cooperate state is more frequent (more than 98%) when 
25<r<S' + l. This is shown in Figure [9 (b)[ This is even more apparent in Figure [TU] which 
shows the contour plotted for the game started with 50% cooperators. In this case, there are 
clearly only two different behaviours, except for some minor border effects. It can be observed 
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(a) When game started with 10% coop- 
erators at random positions. 



(b) When game started with 25% coop- 
erators at random positions. 



Figure 9: Fraction of cooperators on the cycle of length 100 after 10000 generations plotted as 
a contour for varying S and R. 



that the average cooperators after 1000 generations is little less than 100% in some places. But, 
it is clear that the game converges to cooperation before 10000 generations even in these cases. 

Finally, all the plots show some minor effects along the line T = and some noticeable 
effect along the line T = S + 1. This is due to the differences in switching probabilities along 
these borders. Again, this concurs with our analytical results in Section [3. 1.41 that cooperators 
and defectors can coexist in these cases. 



(a) After 1000 generations. 



(b) After 10000 generations. 



Figure 10: Fraction of cooperators on the cycle of length 100 plotted as a contour for varying 
S and R, when the game is started with 50% cooperators at random positions. 



5.2 Complete graph 

For the complete graph, as shown in Section 31 the exact number of cooperators present at the 
beginning of the game determines whether the game converges to defection or cooperation in 
the quadrant SH. For this reason, each simulation was started with a fixed number of players. 
The result obtained are plotted as a contour for varying S and T in Figure 1111 

Here again, the results agree with our rigorous analysis. In SG region, the game did not reach 
a steady state even after 10000 steps. Investigation of the data reveals that the percentage of the 
cooperator after 10000 steps is close to the value of 9* obtained in the analysis (e.g. 6* — 0.49 
when n = 100, T = 1.5, and S — 0.5). Also, it is clear that the initial cooperator percentage 
determines the line in SH region which divides the region with the all-cooperate steady state 
from the region with the all-defect steady state (e.g. the line is T « 5 + 1 when n = 100 and 
^0 = 0.5). 

Finally, we note that Santos et al. [15] also produced simulations for complete graphs. Our 
simulations agree with theirs. 
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(a) Initial cooperators = 25%. (b) Initial cooperators = 50%. (c) Initial cooperators — 75%. 

Figure 11: Fractions of cooperators after 10000 generations on the complete graph of size 100 
with different initial configurations plotted as a contour for varying S and R. 

6 Discussion 

We have studied the imitation update rule in two extreme cases of graph topology. First, for 
the cycle, we have proved that all games converge to either cooperation or defection fast. More 
precisely, if T > 5* + 1, the games converge to the all-defect state fast; and if T < 5* + 1, the 
games converge to the all-cooperate state fast. Even within these regions, the convergence rate 
for the games is different for different values of the payoffs. It is notable from the analytical 
results that the closer the point (5, T) is to the line T = S + 1, the slower the convergence. The 
fact that the cooperators cannot form a barrier (or a cluster as Nowak and May [9] call it) in PD 
region seems to suggest that one dimensional graphs cannot help the evolution of cooperation 
through network reciprocity. 

We highlight the fact that, for the complete graph also, all four games converge to cooperation 
or defection. But the rate of convergence is exponentially slow in n for the SG game. We 
note that experimental studies, including [9l[T5], have wrongly concluded that cooperators and 
defectors can coexist indefinitely. Our results show that this is true for the complete graph only 
for some very particular values of S and T. In the SG region, as we have proved, the number of 
cooperators oscillates around a value, 6* , for an exponential amount of time. We showed that 
convergence to cooperation is more likely after an exponential time if 6* is closer to n, and the 
convergence to defection is more likely if 5* is closer to 0. For some special values of S and T, 
where 6* is an integer, 6* cooperators is a steady state. In this state, cooperators and defectors 
earn equal payoff. We have proved that, for these special values of S and T, the convergence to 
this steady state happens fast. 

7 Conclusions and open problems 

We have shown that indefinite coexistence of cooperators and defectors is impossible on the 
cycle, except for some special values of S and T. More precisely, the coexistence is only possible 
for games where T = 5 + 1 or T = (5 + l)/2. Furthermore, we have shown that, for all the games 
studied, a steady state is reached for cycles in polynomial time. That is, cooperation emerges 
rapidly when 5 -|- 1 > T and T ^ {S + l)/2; defection emerges rapidly when S + 1 <T; and, a 
steady state with cooperators and defectors is reached rapidly when T = S+1 and T = (5+l)/2. 
We also analysed the imitation strategy on complete graphs. The analysis reveals that defection 
emerges fast for Prisoner's Dilemma game, and cooperation emerges fast for Harmony game. 
In the Stag Hunt game, either cooperation or defection emerges fast depending on the initial 
proportion of cooperators. In the Snowdrift game, a metastable state is reached fast. In this 
state, the proportion of cooperators fluctuates around a fixed value for exponential time, before 
converging to cooperation or defection. 

It remains as an open question whether there are graphs other than the cycle on which 
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cooperators and defectors cannot coexist. An interesting extension of this work would be to 
study rigorously the imitation strategy on other graphs, such as trees and grids. In particular, 
based on simulations presented in |14| . regular lattices seem to show very similar, if not the 
same, behaviour to the complete graphs for the whole ST region. But it does not appear that 
a similar analysis can be used to prove this. 

References 

[1] M. Dyer, L. A. Goldberg, C. Greenhill, G. Istrate, and M. Jerrum, Convergence of the 
iterated prisoner's dilemma game, Comb. Probab. Comput. 11 (2002), 135-147. 

[2] W. Feller, An introduction to probability theory and its applications, Wiley, 1968. 

[3] J. Hofbauer and K. Sigmund, Evolutionary games and population dynamics, Cambridge 
University Press, 1998. 

[4] D. M. Kilgour and N. M. Fraser, A taxonomy of all ordinal 2x2 games. Theory and 
Decision 24 (1988), 99-117. 

[5] J. E. Kittock, Emergent conventions and the structure of multi-agent systems, Proceedings 
of the 1993 Santa Fe Institute Complex Systems Summer School, 1993. 

[6] A. N. Licht, Games commissions play: 2x2 games of international securities regulation, 
Yale Journal of International Law 24 (1999), 61-125. 

[7] M. W. Macy and A. Flache, Learning dynamics in social dilemmas. Proceedings of the 
National Academy of Sciences of the United States of America 99 (2002), 7229-7236. 

[8] E. Mossel and S. Roch, Slow emergence of cooperation for win-stay lose-shift on trees, Mach. 
Learn. 67 (2007), 7-22. 

[9] M. A. Nowak, Five rules for the evolution of cooperation. Science 314 (2006), 1560-1563. 

[10] M. A. Nowak and R. M. May, Evolutionary games and spatial chaos, Nature 359 (1992), 
826-829. 

[11] H. Ohtsuki and M. A. Nowak, Evolutionary games on cycles, Proceedings of the Royal 
Society B: Biological Sciences 273 (2006), 2249-2256. 

[12] J. M. Pacheco and F. C. Santos, Network dependence of the dilemmas of cooperation, AIP 
Conference Proceedings 776 (2005), 90-100. 

[13] C. P. Roca, J. A. Cuesta, and A. Sanchez, Imperfect imitation can enhance cooperation, 
Europhysics Letters 87 (2009), 48005-+. 

[14] , Promotion of cooperation on networks? the myopic best response case, The Euro- 
pean Physical Journal B: Condensed Matter and Complex Systems 71 (2009), 587-595. 

[15] F. C. Santos, J. M. Pacheco, and T. Lenaerts, Evolutionary dynamics of social dilemmas 
in structured heterogeneous populations, PNAS 103 (2006), 3490 - 3494. 

[16] F. C. Santos, J. F. Rodrigues, and J. M. Pacheco, Graph topology plays a determinant role 
in the evolution of cooperation. Proceedings of the Royal Society B: Biological Sciences 273 
(2006), 51-55. 

[17] K. H. Schlag, Why imitate, and if so, how?: A boundedly rational approach to multi-armed 
bandits. Journal of Economic Theory 78 (1998), 130 - 156. 

[18] D. Snidal, Relative gains and the pattern of international cooperation. The American Po- 
litical Science Review 85 (1991), 701-726. 



32 



